Wait a few minutes and then try to start the file server again. During that time, memory that was allocated to the file server will be deallocated, freeing enough memory to let you start the server again.

Related topic

Solving problems with file services